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Scientific  Progress 

We  are  trying  to  combine  data  collected  from  multiple  low-cost  sensors  co-located  in  a  dense  sensor  network  to  improve 
classification  result  in  event-detection  applications.  In  other  words,  we  take  advantage  of  highly-correlated  and  coupling 
information  from  multiple  different,  yet  co-located,  sources  (sensors)  recording  the  same  physical  event,  to  improve  the  overall 
detection/classification  accuracy. 

The  main  challenge  that  we  face  in  this  project  comes  from  noises  in  the  data  collected  from  low-cost  low-power  primitive 
sensors.  Sometimes,  there  is  an  outrageous  level  of  noise  (amplitude-wise,  duration-wise,  bandwidth-wise...).  In  fact,  we 
occasionally  stumble  into  sensors  that  completely  failed  during  the  entire  data  collection  process.  We  have  to  deal  with  outliers 
and  missing/incomplete  data  as  well.  Another  major  challenge  is  that  sensors  are  multi-modal  and  heterogeneous.  How  can  we 
properly  fuse  information  in  this  complicated  set-up?  Finally,  the  availability  (or  lack  thereof)  of  well-labeled  data  sets  is  also  a 
major  issue. 

We  are  able  to  take  advantage  of  a  few  recent  ideas  from  the  compressed  sensing  /  sparse  representation  community  -  namely 
joint-sparsity,  group-sparsity,  and  the  sparse-representation-based  classification  framework  -  and  successfully  combine  them 
with  two  of  our  own  novel  ideas  -  optimization  with  class-specific  priors  and  modeling  structured  noise  /  interference  with  a  low- 
rank  model.  Our  expertise  in  dictionary  design,  compressed  sensors  design,  and  optimization  in  sparse  recovery  also  helps.  We 
are  able  to  advance  the  state  of  the  art  classification  accuracy  by  10-15%  in  a  very  challenging  data  sets  from  an  automatic 
border  patrol  application. 
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Dr.  Nasrabadi  and  Lam  Nguyen  are  active  collaborators.  We  have  published  together  multiple  times  on  this  project  over  the 
past  3  years. 

Dr.  Sullivan,  Dr.  Kwon  and  Dr.  Tran-Luu  gave  us  data  sets  relevant  to  this  project:  ultra  wideband  synthetic  aperture  radar  data 
sets,  hyperspectral  data  sets,  and  multi-sensor  acoustic  data  sets.  We  are  actively  collaborating  even  now  after  the  project  has 
ended.  Our  collaborators  above  have  access  to  our  algorithm  designs,  source  codes,  and  published  papers. 


In  our  research,  we  propose  various  novel  collaborative  sparse  representation  methods  for  multi-sensor 
classification  problem,  which  take  into  account  correlations  as  well  as  complementary  information  between 
heterogeneous  sensors  simultaneously  while  considering  joint  sparsity  within  each  sensor’s  observations. 
We  also  robustify  our  models  to  deal  with  the  presence  of  sparse  noise  and  low-rank  interference  terms. 
Especially,  we  observe  that  incorporating  the  noise  or  interfered  signal  as  a  low-rank  component  is  essential 
in  a  multi-sensor  problem  when  multiple  co-located  sources/sensors  simultaneously  record  the  same  phys¬ 
ical  event.  Essentially,  our  proposal  combines  the  strengths  of  multiple  ideas:  (i)  incorporating  related 
information  from  different  sources  (sensors)  to  achieve  an  improvement  in  the  classification  performance, 
(ii)  extracting  and  suppressing  a  large,  dense  and  correlated  (hence  low-rank)  signal/noise  interference 
normally  appeared  in  multi-sensor  data,  and  (iii)  exploiting  prior  structure  in  sparsity  representations  for 
efficiency  and  robustness.  Our  associated  data  model  is: 

min  T s(A)  +  Xl  ||L||# 
s.t.  Ym  =  DmAm  +  Lm  (m  =  1, ...,  M) 


where  M  is  the  total  number  of  sensors  with  M  corresponding  sparsifying  dictionaries  D1,  D2, ...,  DM; 
Y  =  [Y1,  Y2, ...,  Ym]  is  the  concatenated  set  of  measurements  where  each  sample  subset  Ym  collected 
from  the  sensor  m  (rn  =  1, ...,  M )  consists  of  T  observations  Ym  =  [y™,  y™,  •••,  y ™];  A  =  [A1,  A2, ...,  AM] 
contains  the  sparse  codes  with  certain  sparsity  structure;  and  L  =  [L1,  L2, ...,  LM]  is  the  low-rank 
noise/interference.  The  nuclear  matrix  norm  || L|| # ,  defined  as  the  sum  of  all  singular  values  of  the  matrix 
L:  IjLlI*  =  Yhiaii L),  is  a  convex-relaxed  surrogate  of  the  rank  [1,  2];  while  Fs( A)  is  a  convex  struc¬ 
tured  sparsity-promoting  function  that  incorporates  collaborative  structured-sparsity  constraints  both 
within  each  sensor  and  across  multiple  sensors;  and  Xl  >  0  is  a  weighting  parameter  balancing  the  two 
regularization  terms. 

It  is  noted  that  the  model  (1)  is  different  from  our  previous  model  [3]  when  we  developed  the  multi¬ 
sensor  joint  sparse  representation  fusion  model  in  the  presence  of  gross  but  sparse  noise  penalized  by  an 
1 1  -norm  regularization.  In  our  current  approach,  we  study  another  critical  corruption  case:  the  dense 
and  large  but  correlated  noise,  so-termed  low-rank  interference  L.  This  scenario  is  normally  observed 
when  the  recorded  data  is  the  superimpositions  of  target  signals  with  interferences  which  can  be  signals 
from  external  sources  (such  as  a  car  running  through,  a  helicopter  flying  nearby,  or  any  radio- frequency 
interference),  the  underlying  background  that  is  inherently  anchored  in  the  data,  or  any  pattern  noise 
that  remains  stationary  during  signal  transmission.  These  interferences  normally  have  correlated  structure 
and  appear  as  a  low-rank  signal- interference/noise.  In  a  veritable  manner,  the  model  with  the  low-rank 
interference  may  be  more  appropriate  for  the  multi-sensor  dataset  since  the  sensors  are  spatially  co-located 
and  data  samples  are  temporally  recorded,  thus  any  interference  from  external  sources  will  affect  similarly 
on  all  the  multiple  sensor  measurements,  hence  justifying  the  low-rank  property. 

In  order  to  exploit  complementary  features  from  multiple  measurements,  we  incorporate  different 
structures  on  the  concatenated  coefficient  matrix  A  through  the  penalized  function  J-$( A)  which  can 
yield  in  element- wise-sparse,  row-sparse  or  group-sparse  within  each  sensor  and  across  multiple  sensors,  or 
any  hierarchical  tree-sparsity  structure  that  simultaneously  penalize  several  sparsity  levels  in  a  combined 
cost  function.  In  the  most  general  form,  our  model  searches  for  the  group-and-row  sparse  structure 
representation  among  all  sensors  and  low-rank  interference  simultaneously  and  is  termed  as  multi-sensor 

group-joint  sparse  representation  with  low-rank  interference  (MS-GJSR+L): 

C 


min  llAl!i,«j  +  XgY1  IIac|If  +  xl  HlL 

’  c=  1 

s.t.  Ym  =  DmAm  +  Lm  (m  =  1, ...,  M) 


(2) 


where  Ac  =  [Aj,  A2, ...,  A^]  is  the  concatenation  of  sub-coefficient  matrices  A(,n’s  of  all  sensor  rn’ s 
(m  =  1  ,...,M)  induced  by  the  labeled  indexes  corresponding  to  class  c;  and  Ag  >  0  is  the  weighting 
parameter  of  the  group  constrain.  The  optimization  of  (2)  can  be  interpreted  as  follows:  the  first  term 
|i A || i  q  with  q  >  1  is  a  norm  calculated  by  taking  an  t'g-norm  across  the  rows  (observations)  and  then 
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Sensors 
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s8 

s9 

Si-2 

S5-7 

Si— 4 

Si— 7 

Sl-2,5-9 

S1-9 

Table  1:  List  of  sensor  combinations. 


an  1 1  -norm  along  the  columns,  hence  encourages  row-wise  sparsity  within  and  among  all  sensors;  the 
group  regularizer  defined  by  the  second  term  tends  to  minimize  the  number  of  active  groups  in  the 
same  coefficient  matrix  A;  and  the  third  term  penalizes  the  nuclear  norm  of  the  interference  as  afore- 
discussed.  In  succession,  the  model  promotes  a  two-sparsity-level  model:  group-sparse  and  row-sparse  in 
the  combined  coefficient  matrices,  in  parallel  with  extracting  the  low-rank  interference  appearing  in  all 
measurements  all  together.  We  also  propose  a  fast  and  efficient  algorithm  based  on  alternative  direction 
method  to  solve  for  (2)  and  its  convergence  to  optimal  solution  is  guaranteed. 

The  optimization  (2)  is  a  general  framework  that  can  be  simplified  to  introduce  other  methods.  In 
fact,  if  we  let  AG  =  0  then  (2)  becomes  multi-sensor  joint  sparse  representation  with  low-rank  interference 
(MS-JSR+L)  framework  that  extracts  the  low-rank  approximation  in  L  while  promoting  sparsity  at  row 
level  in  the  concatenated  matrix  A  at  the  same  time.  Furthermore,  if  we  eliminate  the  presence  of  L  (i.e. 
set  L  to  be  a  zero  matrix  in  all  optimization  iteration),  then  it  reduces  to  the  MS-JSR  framework  where 
a  joint-sparse  constrain  is  advocated  through  out  all  sensors  without  taking  care  of  the  interfered  noise. 
Finally,  if  the  number  of  sensor  is  further  set  to  M  =  1,  we  simply  have  a  joint-sparse  representation  with 
a  single  sensor  alone. 

Model  (2)  can  even  be  further  extended  into  kernelized  models  which  relies  on  sparsely  representing  a 
test  sample  in  terms  of  all  the  training  samples  in  a  feature  space  induced  by  a  non-linear  kernel  function. 
The  kernelized  model  of  (2)  (namely  MS-KerGJSR+L)  can  be  written  as: 

)IAIIi„  +  AGEIIA^  +  Al||L,|L  ^ 

s.t.  4>(Ym)  =  4>(D~m)Am  +  L (m  =  1, ...,  M) 

where  $  is  an  implicit  mapping  that  maps  any  set  of  vectors  onto  a  higher  dimensional  space,  possibly 
infinite,  and  L ^  is  the  additive  low-rank  interference  in  the  kernal  feature  domain.  Note  that  in  general 
the  mapping  function  is  not  explicitly  defined,  but  rather  characterized  by  the  kernel  function  k.  defined 
as  the  inner  product  of  two  vectors:  k(xj,  Xj)  =  (4>(xj),  <F(xj)).  Commonly  used  kernels  include  the  radial 
basis  function  (RBF)  Gaussian  kernel  /c(xj,Xj)  =  exp(—  ||xj  —  Xj||2  /f?2)  with  rj  used  to  control  the  width 
of  the  RBF,  and  order-d  polynomial  kernels  ft(xj,Xj)  =  (x,;  •  Xj  +  l)rf  [4,  5].  The  algorithm  to  solve  for 
(3)  is  modified  from  the  algorithm  of  (2)  with  attentive  manipulations  of  involving  kernel  functions. 

We  verify  the  effectiveness  of  the  proposed  methods  via  solving  a  classification  problem  on  multi-sensor 
data  focusing  on  discriminating  between  human  and  non-human  footsteps.  The  experimental  setup  is  as 
follows:  a  set  of  nine  sensors  including  four  acoustic  (<Si_4),  three  seismic  (55-7),  one  passive  infrared 
(PIR)  (Ss),  and  one  ultrasonic  ( Sg )  sensors  are  used  to  measure  the  same  physical  event  simultaneously  on 
the  field.  The  ultimate  goal  is  to  detect  whether  the  event  involves  human  or  human  and  animal  footsteps. 
The  system  allows  to  evaluate  the  classification  results  based  on  various  combinations  of  recording  sources, 
not  just  multiple  sensors  of  the  same  brands  but  also  sensors  of  different  signal  types.  For  all  methods, 
15  combination  sets  of  sensors  are  processed  and  compared,  in  which  the  first  9  sets  are  conducted  using 
one  single  sensor  separately,  corresponding  to  Si,  S2,  ...  Sg.  The  next  6  sets  combine  multiple  sensors 
into  various  scenarios  as  listed  in  Table  1. 

Our  six  proposed  methods,  which  are  based  on  different  assumptions  of  the  structures  of  coefficient 
vectors,  noise/interference  and  linearity  properties,  are  processed  through  all  15  sensor  sets  to  determine 
the  joint  coefficient  matrix  A  and  class  label  is  then  determined  by  minimal  error  residua  classifiers. 
The  results  are  then  compared  with  popular  powerful  techniques  such  as  sparse  logistic  regression  (SLR), 
heterogeneous  feature  machine  (HFM),  linear  support  vector  machine  (SVM),  and  their  kernelized  versions 
to  verify  the  effectiveness  of  the  proposed  methods.  The  classification  rates  defined  as  ratios  of  the  total 
number  of  correctly  classified  samples  to  the  total  number  of  testing  samples,  expressed  as  percentages,  are 
plotted  in  Fig.  1  and  Fig.  2,  corresponding  to  testing  data  collected  in  December  09  and  10,  respectively. 
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Figure  1:  Comparison  of  classification  results  -  DEC09 


Figure  2:  Comparison  of  classification  results  -  DEC10 


We  tabulate  the  classification  performance  of  all  proposed  models  as  well  as  competing  methods  in  Table 
2.  The  second  and  third  columns  in  each  table  describe  the  classification  accuracy  by  using  single  sensor 
and  multiple  sensors  (which  average  the  rates  of  sets  1-9  and  10-15,  respectively),  and  the  last  column 
shows  the  overall  results  by  averaging  over  all  15  sensor  sets. 

The  experimental  results  with  particular  datasets  show  that  our  proposed  models  outperform  the 
other  conventional  classifiers.  These  results  also  reveal  several  critical  orservations:  (1)  the  use  of  com¬ 
plementary  information  from  multiple  sensors  significantly  improves  the  classification  results  over  just 
using  a  singular  sensor;  (2)  appropriate  structured  regularizations  (joint  and  group  sparsity)  bring  more 
advantage  in  selecting  the  right  classes,  hence  increase  the  accuracy  rate  of  classification  results;  (3) 
low-rank  noise  is  a  critical  issue  in  multi-sensor  fusion  problem  and  (4)  the  classification  in  feature  space 
induced  by  a  kernel  function  yields  a  compelling  performance  improvement.  Our  proposed  methods  not 
only  provide  new  tools,  but  also  deepen  the  understanding  of  adaptive  sparsity  modeling,  signal  behavior 
and  efficient  multi-sensor  data  collection  and  collaboration.  Nevertheless,  although  our  techniques  are 
designed  for  border  patrol  control  in  military  purposes,  they  are  not  restricted  to  this  specific  application. 
Rather,  they  can  be  applied  to  any  set  of  classification  or  discrimination  problems,  where  the  data  is 
usually  collected  from  multiple  sources. 
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